A Hierarchical Ensemble of Decision Trees Applied to Classifying Data from a Psychological Experiment

from a psychological experiment Yannick Lallement Human-Computer Interaction Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 [email protected] Abstract Classifying by hand complex data coming from psychology experiments can be a long and di cult task, because of the quantity of data to classify and the amount of training it may require. One way to alleviate this problem is to use machine learning techniques. We built a classi er based on decision trees that reproduces the classifying process used by two humans on a sample of data and that learns how to classify unseen data. The automatic classi er proved to be more accurate, more constant and much faster than classi cation by hand. Introduction Classi cation of complex data coming from psychological experiments is an important issue in cognitive psychology. Such classi cation is often done by two or more persons who subjectively rate the human behavior. This process can be long and labor-intensive, and there is often too much data to be classi ed by humans in a reasonable amount of time. Moreover, the psychology community considers the classi cation acceptable if the inter-rater reliability between the persons is at least 80%, which, given the variability of human data, often demands a lot of training. We encountered such a problem regarding the classi cation of human behavior over time. We built a learning classi er to make the classi cation faster and more reliable. In the following, we describe our data, the classi er we built, and the results we obtained. The KA-ATC c Task In the Kanfer-Ackerman Air Tra c Control c task (Ackerman & Kanfer 1994) is used to study problemsolving and learning in a dynamic environment. When performing this task, participants are presented with 0Copyright c 1998, American Association for Arti cial Intelligence (www.aaai.org). All rights reserved. .... FLT# TYPE SCHEDULED FUEL POS. TIME : 1:00 --------------------Score : 1000 -> 431 DC10 1:35 6 3 n Landing Pts: 1100 Penalty Pts: 100 268 747 1:45 6 3 s Runways : DRY 555 prop 1:50 6 3 e Wind : 10 20 knots from SOUTH 111 747 2:00 6 3 w 912 727 1:15 6 2 n Flts in Queue: 67 prop 1:05 6 2 s to accept 113 727 1:20 6 2 e 2 w 157 DC10 1:20 6 1 n 872 727 1:10 6 1 s 1 e 222 prop 1:00 5 1 w